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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 
Mi claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
of the Office or upon petition by the applicant. See 37 CFR 1.313 and MPEP 1308. 

1 . ^ This communication is responsive to amendment filed 2/24/05 . 

2. The allowed claim(s) is/are 1.4-10.13-19 and 22-38 . 

3. ^ The drawings filed on 13 July 2001 are accepted by the Examiner. 

4. □ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a) □ All b) □ Some* c) □ None of the: 

1 . □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 

3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). 
* Certified copies not received: . 

Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
THIS THREE-MONTH PERIOD IS NOT EXTENDABLE. 

5. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 

INFORMAL PATENT APPLICATION (PTO-152) which gives reason(s) why the oath or declaration is deficient. 

6. □ CORRECTED DRAWINGS ( as "replacement sheets") must be submitted. 

(a) □ including changes required by the Notice of Draftsperson's Patent Drawing Review ( PTO-948) attached 

1) □ hereto or 2) □ to Paper NoVMail Date . 

(b) □ including changes required by the attached Examiner's Amendment / Comment or in the Office action of 

Paper NoVMail Date . 

Identifying indicia such as the application number (see 37 CFR 1.84(c)) should be written on the drawings in the front (not the back) of 
each sheet. Replacement sheet(s) should be labeled as such in the header according to 37 CFR 1.121(d). 

7. □ DEPOSIT OF and/or INFORMATION about the deposit of BIOLOGICAL MATERIAL must be submitted. Note the 

attached Examiner's comment regarding REQUIREMENT FOR THE DEPOSIT OF BIOLOGICAL MATERIAL. 
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Paper NoVMail Date 
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Reasons for Allowance 

1 . The following is an examiner's statement of reasons for allowance: The 
amendment filed 2/24/05, in light of applicants' comments, places the application into 
condition for allowance by incorporating into all of the independent claims that the intent 
determination comprises resolving referential ambiguity associated with the user(s) and 
device(s) in the environment based on at least a portion of the received multi-modal 
data. As shown in the specification and brought out specifically in applicants' 
comments, the referential ambiguity techniques which distinguish and resolve between 
multiple sensors and inputs of users regarding different actions to be taken among 
various devices, are not present in the Geddes and Strubbe references. Azvine et al 
may show various ambiguity resolving techniques between different inputs, but still do 
not show the flexibility and broad ability of the techniques as brought out in the present 
invention, especially in combination with the focus, mood, and intent determination 
features including storing the results for future determinations and executing actions in 
the environment accordingly, in the multi-modal conversational computing system of the 
present invention. The features combined of the present invention, as brought out in 
the amendment filed 2/24/05, are not set forth in the prior art of record. 

Independent claims: 1 amended - system. 4 amended - system wherein the 
execution of the actions in the environment effects the intent, focus, and mood of the 
users. 5 amended - system wherein the execution of the actions controls the devices 
to request further input to assist in making the determinations. 6 amended - system 
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wherein the execution of the actions initiates a process to complete or correct or 
disambiguate what the system understands from previous input. 7 amended - system 
wherein the processor abstracts the received input into one or more events prior to 
making the determinations. 8 amended - system wherein the processor performs 
recognition operations on the input prior to making the determinations. 9 amended - 
system with features of both claims 7 amended and 8 amended. 10 amended - method 
corresponding to claim 1 amended. 13 amended - method corresponding to claim 4 
amended. 14 amended - method corresponding to claim 5 amended. 15 amended - 
method corresponding to claim 6 amended. 16 amended - method corresponding to 
claim 7 amended. 17 amended - method corresponding to claim 8 amended. 18 
amended - machine readable medium corresponding to claim 1 amended. 19 
amended - system like claim 1 amended and specifying the audio and image data. 29 
amended - system like claim 19 amended wherein the execution of the actions in the 
environment effects the intent, focus, and mood of the users. 30 amended - system 
like claim 19 amended wherein the execution of the actions controls the devices to 
request further input to assist in making the determinations. 31 amended - system like 
claim 19 amended wherein the processor abstracts the received input into one or more 
events prior to making the determinations. 32 amended - system like claim 19 
amended wherein the processor performs recognition operations on the input prior to 
making the determinations. 36 amended - system like claim 19 amended wherein the 
execution of the actions initiates a process to complete or correct or disambiguate what 
the system understands from previous input. 37 amended - system with features of 
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both claims 31 amended and 32 amended. 38 amended - method corresponding to 
claim 19 amended. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

2. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Steven P Sax whose telephone number is (571 ) 
272-4072. The examiner can normally be reached on Monday thru Friday, 8:30 AM - 
5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid can be reached on (571 ) 272-4063. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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